Papers with English sentences
Should you marginalize over possible tokenizations? (2023.acl-short)
Copied to clipboard
| Challenge: | Autoregressive language models map token sequences to probabilities. |
| Approach: | They propose to marginalize over all tokenizations to compute the probability of a string . they find that the gap in log-likelihood is no larger than 0.5% in most cases . |
| Outcome: | The proposed algorithm shows that the gap in log-likelihood is no larger than 0.5% in most cases . but it becomes more pronounced for data with long complex words, the authors show . |
Dependency Tree Annotation with Mechanical Turk (D19-59)
Copied to clipboard
| Challenge: | a recent study shows that crowdsourcing is often used to obtain linguistic annotations but is rarely used for parsing. |
| Approach: | They propose to use Mechanical Turk to crowdsource parse trees using an interactive graphical dependency tree editor. |
| Outcome: | The proposed method is the first published use of Mechanical Turk to crowdsource parse trees . the authors find that the workers achieve high levels of accuracy on 72% of the sentences . |
GEM: Generative Enhanced Model for adversarial attacks (D19-66)
Copied to clipboard
| Challenge: | Our model generated malicious claims that mixed facts from various articles, so it became difficult to classify their truthfulness. |
| Approach: | They used a model that inherited the knowledge of pretrained GPT-2 to generate controlled sentences with some additional control. |
| Outcome: | The proposed model generated malicious claims that mixed facts from Wikipedia articles, making it difficult to classify their truthfulness. |
Realizing Universal Dependencies Structures (D19-63)
Copied to clipboard
| Challenge: | UD-SURFR is a symbolic surface realizer for Universal Dependencies structures . it is based on a tree of constituents that is then converted into an English sentence . |
| Approach: | They propose a symbolic system that transforms a dependency tree into a tree of constituents that is transformed into an English sentence by an existing realizer. |
| Outcome: | The proposed system is quite fast and showed competitive results for English sentences using automatic and manual evaluation measures. |
xSIM++: An Improved Proxy to Bitext Mining Performance for Low-Resource Languages (2023.acl-short)
Copied to clipboard
| Challenge: | xsim++ provides a reliable proxy for bitext mining without expensive pipelines. |
| Approach: | They propose a new proxy proxy based on similarity in a multilingual embedding space . they validate this proxy by running a significant number of bitext mining experiments for a set of low-resource languages and then train NMT systems on the mined data. |
| Outcome: | The proposed proxy improves on xsim++ and trains on the mined data. |
ZuCo 2.0: A Dataset of Physiological Recordings During Natural Reading and Annotation (2020.lrec-1)
Copied to clipboard
| Challenge: | a new dataset of eye-tracking and electroencephalography captures language understanding . eye movement data provides millisecond-accurate records of where humans look when reading . |
| Approach: | They recorded and preprocessed eye-tracking and electroencephalography data during natural reading and during annotation. |
| Outcome: | The study combines eye-tracking and electroencephalography to capture the reading process . the data can be used to evaluate state-of-the-art machine learning systems . |
CCG Parsing Algorithm with Incremental Tree Rotation (N19-1)
Copied to clipboard
| Challenge: | Combinatory Categorial Grammar (CCG) is a mildly context sensitive grammar formalism that excels in incremental sentence processing. |
| Approach: | They propose a new incremental parsing algorithm that uses a syntactic approach . it uses right-branching constituent structures and optional constituents that adjoin on the right . |
| Outcome: | The proposed algorithm can cover the whole CCGbank with greater incrementality and accuracy than previous proposals. |
Automatic Question Generation using Relative Pronouns and Adverbs (P18-3)
Copied to clipboard
| Challenge: | Automatic Question Generation is a system that generates multiple, natural language questions using relative pronouns and relative adverbs from complex English sentences. |
| Approach: | They propose a system that automatically generates multiple, natural language questions using relative pronouns and relative adverbs from complex English sentences. |
| Outcome: | The proposed system generates multiple, natural language questions using relative pronouns and relative adverbs from complex English sentences. |
Extracting Latent Steering Vectors from Pretrained Language Models (2022.findings-acl)
Copied to clipboard
| Challenge: | Prior work on controllable text generation has focused on learning how to control language models through trainable decoding, smart-prompt design, or fine-tuning based on a desired objective. |
| Approach: | They propose to extract latent vectors directly from pretrained language model decoders without fine-tuning. |
| Outcome: | The proposed approach generates a target sentence nearly perfectly for English sentences . it outperforms pooled hidden states of models on a textual similarity benchmark . |
When classifying grammatical role, BERT doesn’t care about word order... except when it matters (2022.acl-short)
Copied to clipboard
| Challenge: | Recent work has shown large language models are surprisingly word order invariant . however, word order knowledge is crucial in defining later-layer representations of words . |
| Approach: | They probe grammatical role representations in English BERT and GPT-2 to find word order crucial . they find word orders are crucial in defining later-layer representations of words in non-prototypical positions . |
| Outcome: | The proposed model is based on natural prototypical inputs where word order is crucial for correct classification. |
The Influence of Context on Sentence Acceptability Judgements (P18-2)
Copied to clipboard
| Challenge: | a paper examining the influence of document context on acceptability judgements for English sentences is published in journal journal of linguistics. |
| Approach: | They propose to use document context to assess acceptability judgements for English sentences . they also test the accuracy of neural models that incorporate document context during training . |
| Outcome: | The proposed model improves acceptability ratings for ill-formed sentences, but reduces them for well-formed ones. |
A Multilingual Wikified Data Set of Educational Material (L18-1)
Copied to clipboard
Iris Hendrickx, Eirini Takoulidou, Thanasis Naskos, Katia Lida Kermanidis, Vilelmini Sosoni, Hugo de Vos, Maria Stasimioti, Menno van Zaanen, Panayota Georgakopoulou, Valia Kordoni, Maja Popovic, Markus Egg, Antal van den Bosch
| Challenge: | a crowdsourcing effort to annotate and link parallel texts has been unsuccessful . a data set of parallel texts in eleven languages is presented . |
| Approach: | They present a wikified data set of English sentences linked to Wikipedia pages . they use crowdsourcing to annotate the texts and perform crowdsourcing for complex annotations . |
| Outcome: | The proposed data set is valuable as it constitutes a rich resource . it includes annotated data of English sentences linked to translations in eleven languages . |
Language Models Largely Exhibit Human-like Constituent Ordering Preferences (2025.naacl-long)
Copied to clipboard
| Challenge: | English sentences are typically inflexible vis-à-vis word order, but constituents show far more variability in ordering. |
| Approach: | They compare LLMs with four types of constituent movement to evaluate their performance on heavy NP shift, particle movement, dative alternation, and multiple PPs. |
| Outcome: | The proposed model performs well on four types of constituent movement: heavy NP shift, particle movement, dative alternation, and multiple PPs. |
Targeted Syntactic Evaluation of Language Models (D18-1)
Copied to clipboard
| Challenge: | Recent advances have led to an explosion of neural network-based LM architectures. |
| Approach: | They propose to supplement perplexity with a metric that assesses whether a language model can predict the grammatical sentence more accurately than an ungrammatically-based model. |
| Outcome: | The proposed model performed poorly on many of the constructions. |
Evaluating Gender Bias in Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Using morphological analysis, we find that MT models exhibit gender-biased translation errors when training data encode stereotypes not relevant for the task. |
| Approach: | They propose an automatic gender bias evaluation method for eight target languages with grammatical gender based on morphological analysis. |
| Outcome: | The proposed method is based on two recent coreference resolution datasets composed of English sentences cast participants into non-stereotypical gender roles. |
Alignment-Augmented Consistent Translation for Multilingual Open Information Extraction (2022.acl-long)
Copied to clipboard
| Challenge: | Existing training data in English is limited to supervised Open Information Extraction (OpenIE) . |
| Approach: | They propose a model that automatically converts English sentences into other languages by using Alignment-Augmented Constrained Translation (AACTrans) they train a generative OpenIE model that outputs for each sentence relations in the first stage and all extractions containing the relation in the second stage. |
| Outcome: | The proposed model outperforms existing models on Spanish, Portuguese, Chinese, Hindi and Telugu on 5 languages. |
DocAMR: Multi-Sentence AMR Representation and Evaluation (2022.naacl-main)
Copied to clipboard
Tahira Naseem, Austin Blodgett, Sadhana Kumaravel, Tim O’Gorman, Young-Suk Lee, Jeffrey Flanigan, Ramón Astudillo, Radu Florian, Salim Roukos, Nathan Schneider
| Challenge: | Abstract Meaning Representation (AMR) graphs are compared to gold graphs by the Smatch metric, but lack a well-defined representation and evaluation. |
| Approach: | They propose an algorithm for deriving a unified graph representation using a super-sentential annotation method. |
| Outcome: | The proposed algorithm avoids the pitfalls of over-merging and lacks coherence from under merging. |
Probabilistic, Structure-Aware Algorithms for Improved Variety, Accuracy, and Coverage of AMR Alignments (2021.acl-long)
Copied to clipboard
| Challenge: | Current algorithms for AMR parsing suffer from limited coverage and less-than-ideal accuracy . a new algorithm for AML uses unsupervised learning and heuristics to align components of AMR graphs to spans in English sentences . |
| Approach: | They propose algorithms for aligning components of Abstract Meaning Representation graphs to spans in English sentences. |
| Outcome: | The proposed approach covers a wider variety of AMR substructures than previously considered . it achieves higher coverage of nodes and edges, and does so with higher accuracy. |
Grounding language acquisition by training semantic parsers using captioned videos (D18-1)
Copied to clipboard
| Challenge: | a new method for parsing sentences using captioned videos is being developed . we use video clips to ground the semantics of language, but without annotations . |
| Approach: | They develop a semantic parser that is trained in a grounded setting using captioned videos . they use a corpus of sentences paired with videos without other annotations to train it . |
| Outcome: | The proposed parser recovers the meaning of English sentences despite no annotations . learning a grounded semantic parsers can expand the range of data that parseurs can be trained on . |
That was the last straw, we need more: Are Translation Systems Sensitive to Disambiguating Context? (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for translation of ambiguous text use context to disambiguate meaning . current models for MTs consistently translate English idioms literally, whereas LMs are context-aware . |
| Approach: | They use a dataset of 512 pairs of English sentences to study semantic ambiguities . they use literal and figurative idioms to disambiguate intended meaning . |
| Outcome: | The results show that current models translate English idioms literally, even when the context suggests a figurative interpretation. |
Transforming Complex Sentences into a Semantic Hierarchy (P19-1)
Copied to clipboard
| Challenge: | Existing approaches for recursively splitting and rephrasing complex English sentences into a semantic hierarchy of simplified sentences are lacking. |
| Approach: | They propose a method for recursively splitting and rephrasing complex English sentences into a semantic hierarchy of simplified sentences. |
| Outcome: | The proposed approach outperforms state-of-the-art approaches in MT and information extraction tasks. |
What Meaning-Form Correlation Has to Compose With: A Study of MFC on Artificial and Natural Language (2020.coling-main)
Copied to clipboard
| Challenge: | Compositionality is a widely discussed property of natural languages, although its exact definition has been elusive. |
| Approach: | They propose that compositionality can be measured by measuring meaning-form correlation . they analyze three sets of languages: artificial toy languages tailored to be compositional . |
| Outcome: | The proposed method can assess compositionality on three sets of languages . linguistic phenomena such as synonymy and ungrounded stop-words weigh on the results . |
CEFR-Based Sentence Difficulty Annotation and Assessment (2022.emnlp-main)
Copied to clipboard
| Challenge: | Controllable text simplification is a crucial assistive technique for language learning and teaching. |
| Approach: | They propose a sentence-level assessment model to handle unbalanced level distribution . previous studies have suggested that controllable text simplification is difficult to apply . |
| Outcome: | The proposed method outperforms baselines in readability assessment by scoring macro-F1 on the level assessment. |
BiSECT: Learning to Split and Rephrase Sentences with Bitexts (2021.emnlp-main)
Copied to clipboard
| Challenge: | Several past efforts have created Split and Rephrase training sets, which consist of long, complex input sentences paired with multiple shorter sentences that preserve the meaning of the input sentence. |
| Approach: | They propose a new dataset and a model for this task by extracting 1-2 sentence alignments from bilingual parallel corpora and using machine translation to convert both sides of the corpus into the same language. |
| Outcome: | The proposed model can perform a wider variety of split operations and improve upon previous state-of-the-art approaches in automatic and human evaluations. |
The Indigenous Languages Technology project at NRC Canada: An empowerment-oriented approach to developing language software (2020.coling-main)
Copied to clipboard
Roland Kuhn, Fineen Davis, Alain Désilets, Eric Joanis, Anna Kazantseva, Rebecca Knowles, Patrick Littell, Delaney Lothian, Aidan Pine, Caroline Running Wolf, Eddie Santos, Darlene Stewart, Gilles Boulianne, Vishwa Gupta, Brian Maracle Owennatékha, Akwiratékha’ Martin, Christopher Cox, Marie-Odile Junker, Olivia Sammons, Delasie Torkornoo, Nathan Thanyehténhas Brinklow, Sara Child, Benoît Farley, David Huggins-Daines, Daisy Rosenblum, Heather Souter
| Challenge: | This paper describes the first, three-year phase of a project at the National Research Council of Canada that is developing software to assist Indigenous communities in preserving their languages and extending their use. |
| Approach: | They describe the first phase of a project at the National Research Council of Canada that is developing software to assist Indigenous communities in preserving their languages. |
| Outcome: | The proposed software will help Indigenous communities preserve and revitalize their languages and extend their use. |
XED: A Multilingual Dataset for Sentiment Analysis and Emotion Detection (2020.coling-main)
Copied to clipboard
| Challenge: | XED is a multilingual fine-grained emotion dataset for English and other low-resource languages. |
| Approach: | They propose a multilingual fine-grained emotion dataset using Plutchik's Wheel of Emotions and a projection scheme to annotate Finnish and English sentences. |
| Outcome: | The proposed dataset is based on human-annotated Finnish and English sentences and projected annotations for 30 additional languages. |
A Corpus for Reasoning about Natural Language Grounded in Photographs (P19-1)
Copied to clipboard
| Challenge: | a dataset for visual reasoning with natural language and images is available. |
| Approach: | They propose a dataset for joint reasoning about natural language and images . they crowdsource 107,292 examples of English sentences paired with web photographs . |
| Outcome: | The proposed dataset combines 107,292 examples of English sentences with web photographs . Qualitative analysis shows the data requires compositional joint reasoning . |
YASO: A Targeted Sentiment Analysis Evaluation Dataset for Open-Domain Reviews (2021.emnlp-main)
Copied to clipboard
| Challenge: | YASO contains 2,215 English sentences from dozens of review domains, annotated with target terms and their sentiment. |
| Approach: | They propose a new TSA evaluation dataset of open-domain user reviews in English . YASO contains 2,215 English sentences annotated with target terms and their sentiment . |
| Outcome: | The proposed dataset verifies the reliability of the annotations and explores the characteristics of the collected data. |
Open-source Multi-speaker Corpora of the English Accents in the British Isles (2020.lrec-1)
Copied to clipboard
| Challenge: | Using a dataset of high-quality audio, the authors examine the accents of 120 volunteers in the British Isles. |
| Approach: | They present a dataset of high-quality audio of English sentences recorded by volunteers with different accents of the British Isles. |
| Outcome: | The transcribed audio includes pronunciations of global locations, major airlines and common personal names in different accents. |
Towards Unsupervised Recognition of Token-level Semantic Differences in Related Documents (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on word-level predictions and highlighting semantic differences in natural language documents did not focus on semantic differences as the main target. |
| Approach: | They propose to perform a token-level regression task to highlight semantic differences between two documents . they use word alignment and sentence-level contrastive learning to evaluate the approaches . |
| Outcome: | The proposed approach has a robust correlation to gold labels, but all unsupervised approaches leave a margin of improvement. |
Do you Feel Certain about your Annotation? A Web-based Semantic Frame Annotation Tool Considering Annotators’ Concerns and Behaviors (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing tools for manual annotations are resourceintensive and complex, and experienced annotators and tools specialized for the purpose of the annotation task are required. |
| Approach: | They propose to use a web-based application with a responsive design for modular semantic frame annotation (SFA) the proposed application keeps track of the time and changes during the annotation process and stores the users’ confidence with the current annotation. |
| Outcome: | The proposed system can be used to build a manually annotated corpus and its arguments for task 2 of SemEval 2019 regarding unsupervised lexical frame induction. |
D3CODE: Disentangling Disagreements in Data across Cultures on Offensiveness Detection and Evaluation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies on annotator subjectivity focus on Western contexts and only document differences across age, gender, or racial groups. |
| Approach: | They propose a large-scale cross-cultural dataset of parallel annotations for offensive language in over 4.5K English sentences annotated by a pool of more than 4k annotators from 21 countries. |
| Outcome: | The proposed dataset captures annotators’ moral values along six moral foundations: care, equality, proportionality, authority, loyalty, and purity. |
ICL CIPHERS: Quantifying ”Learning” in In-Context Learning via Substitution Ciphers (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies suggest that In-Context Learning operates in dual modes . however, disentangling these modes remains a challenging goal . |
| Approach: | They propose a class of task reformulations based on substitution ciphers borrowed from classic cryptography. |
| Outcome: | The proposed model can solve tasks with a BIJECTIVE mapping, but it requires 'deciphering' the latent cipher. |